Diagnosis at Scale: Detecting the Expertise Level and Knowledge States of Lifelong Professional Learners

Our research is about identifying gaps in the knowledge of professional software developers, as part of an ongoing project to provide tools to support their lifelong learning needs. We developed metrics that when applied to programmers’ online activities in Stack Overflow allowed us to determine the knowledge states of users on specific topics indicating what each user knows they know and their knowledge “gaps”, both what they know they don’t know and what they don’t know they don’t know. Further we were able to find patterns that showed that at all levels of expertise there are still “unknown unknowns”, and these are particularly dangerous since the software professional is unaware of their weaknesses in these areas.